Video camera with electronic picture stabilizer

ABSTRACT

A video camera including a motion detecting circuit in which correlative values of pixels at a present field are evaluated through the comparison of an image signal of a last field and an image signal of the present field. In the motion detecting circuit, an average correlative value and a minimum correlative value are further evaluated. A microcomputer determines whether or not these correlative values satisfy predetermined conditions for each of detection areas defined within an image field. A detection area which satisfies all the predetermined conditions becomes a valid detection area. A whole motion vector is detected by methods different from each other according to the number of the valid detection areas.

BACKGROUND OF THE INVENTION

1. Field of the Invention

The present invention relates to a video camera. More specifically, thepresent invention relates to a compact video camera with an electronicpicture stabilizer, which is utilized as a camcorder, for example.

2. Description of the Related Art

One example of a method for detecting an unintentional motion componentof an image sensing device is disclosed in, for example, the TwentiethImage Engineering Conference in 1989 by Matsushita Electric IndustrialCorporation. In this prior art method, by utilizing a motion vectorobtained by a representative point matching method disclosed in, forexample, Japanese Patent Application Laid open No. 61(1986)-201581 [H04N7/137], the motion component of the image sensing device is detectedaccording to image information. In this prior art, an electronic picturestabilization is performed on the basis of a whole motion vectorobtained from the image information.

More specifically, in this prior art, four detection areas are arrangedin an image field, and four portion motion vectors are obtained from theimage field. Then, the whole motion vector is evaluated by averaging theportion motion vectors of the four detection areas, or the whole motionvector is evaluated by averaging portion motion vectors of two portionmotion vectors having intermediate values of the four portion motionvectors, whereby the electronic picture stabilizer is performed on thebase of the whole motion vector thus evaluated.

However, in such a method, in a case where an object has passed througha detection area, for example, when there is a detection area in which aportion motion vector which is not caused by the unintentional motion ofthe camera is detected, the whole motion vector is affected by thisdetection area, and therefore, the motion of the camera is notaccurately detected, and thus, electronic picture stabilization cannotbe performed with precision.

SUMMARY OF THE INVENTION

Therefore, a principal object of the present invention is to provide avideo camera having a novel electronic picture stabilizer.

Another object of the present invention is to provide a video camerawith an electronic picture stabilizer capable of accurately performingthe electronic picture stabilization.

In a video camera according to the present invention, valid detectionarea determinating means includes first means for evaluating correlativevalues of pixels at a present field or frame on the basis of an imagesignal of a last field or frame and an image signal of the present fieldor frame; means for evaluating an average correlative value that is amean value of the correlative values; means for evaluating a minimumcorrelative value that is a minimum value out of the correlative values;means for evaluating a value obtained by dividing the averagecorrelative value by the minimum correlative value; means for evaluatinga gradient associated with the minimum correlative value; and means fordetermining whether or not a motion vector is correctly detected inaccordance with whether or not all of the following conditions aresatisfied:

(A) the average correlative value>a first threshold value;

(B) the value obtained by dividing the average correlative value by theminimum correlative value>a second threshold value;

(C) the gradient>a third threshold value (when the average correlativevalue≧a fifth threshold value); and

(D) the gradient>a fourth threshold value (when the average correlativevalue<the fifth threshold value), the third threshold value being equalto or larger than the fourth threshold value (the third thresholdvalue≧the fourth threshold value).

As indicated in the above described conditions (C) and (D), thethreshold values to be compared with the gradient are changed inaccordance with the average correlative value. When a contrast in ascreen is large, that is, when the average correlative value is large,the threshold value to be compared with the gradient is set to be largeas indicated by the condition (C), and when the contrast in the screenis small, that is, when the average correlative value is small, thethreshold value to be compared with the gradient is set to be small asindicated by the condition (D). Therefore, the gradient can be comparedwith the threshold value in corresponding to a change of the contrast inthe screen (the third threshold value or the fourth threshold value),and therefore, determination whether or not an object having repeatedpattern exists in the image field is performed more accurately.

Then, it is determined whether or not each of respective detection areasdefined within the screen image field is a valid detection area by thevalid detection area determinating means through determination whetheror not all of the above described conditions are satisfied in respect tothe detection area.

In a case where the number of the valid detection areas is larger than apredetermined threshold value, first, a first absolute value and asecond absolute value are evaluated for each detection area by firstcalculation means and second calculation means, respectively. By addingthe first absolute value and the second absolute value to each other foreach detection area by addition means so as to obtain a firstdispersion, and the arbitrary number of the first dispersions areselected by selection means in order of small. A whole motion vector isdetected on the basis of portion motion vectors of the detection areasrepresenting the selected first dispersions.

When the number of the valid detection areas is less than thepredetermined value, first, a portion motion vector of an arbitraryinvalid detection area is replaced with the whole motion vector of afield or frame before the present field or frame by first replacementmeans. Thereafter, the portion motion vector of the invalid detectionarea that is replaced with the whole motion vector and the portionmotion vectors of the valid detection areas are utilized for detectingthe whole motion vector at the present field or frame according to theabove described processing.

Furthermore, the number of valid detection areas is zero, as the wholemotion vector of the present field or frame, a resulted vector bymultiplying the whole motion vector evaluated one field or frame beforeby a predetermined coefficient less than 1 (decimal number less than 1)is utilized.

In accordance with the present invention, since a detecting method fordetecting the whole motion vector is changed according to the number ofvalid detection areas, even if a portion motion vector that is notcaused by the unintentional motion of the camera is detected inperforming the electronic image stabilization, the influence thereofbecomes very small, and therefore, it becomes possible to accuratelydetect an unintentional motion amount. Therefore, the unintentionalmotion of the camera can be corrected with precision.

It is correctly determined whether or not the object having repeatedpattern (stripe or the like) exists in the image, and a detection areais hardly determined as an invalid detection area even when the contrastin the screen is slightly reduced, and therefore, a detection accuracyof the motion vector is increased.

Furthermore, the whole motion vector may be evaluated on the basis of asecond dispersion. More specifically, first, the second dispersion isevaluated by second dispersion calculation means by utilizing an averagevector of the portion motion vectors which is evaluated by averagingmeans, and the portion motion vectors of the respective detection areas.A fact that the second dispersion is large indicates that variation ofthe portion motion vectors is large. This tendency occurs in a casewhere there is an object which moves in only a portion of the screen. Onthe other hand, a fact that the second dispersion is small means thatthe variation of the portion motion vectors is small. This tendencyoccurs in a case where no moving object exists on the screen, or a casewhere there is an object which moves all over the screen.

Therefore, if the whole motion vector by utilizing the portion motionvector of the case where the second dispersion is large, the wholemotion vector having low reliability is obtained. In contrast, if thewhole motion vector is evaluated by utilizing the portion motion vectorof the case whether the second dispersion is small, a whole motionvector having high reliability is obtained. Therefore, in the case wherethe second dispersion is large, a value of the whole motion vector ismade small by multiplying the average vector of the portion motionvectors by a small coefficient. Furthermore, in the case where thesecond dispersion is small, a value of the whole motion vector is madelarge by multiplying the average vector of the portion motion vectors bya large coefficient. That is, the whole motion vector of the case of thelarge second dispersion is not used as possible, and in contrast, thewhole motion vector of the case of the small second dispersion is usedif possible.

In addition, a degree of a magnitude of the second dispersion isdetermined by comparing means, and the whole motion vector is detectedin accordance with a comparison result.

In accordance with the present invention, by grasping a status of thescreen through evaluation of the second dispersion, the whole motionvector is detected in accordance with the magnitude of the seconddispersion, and therefore, it is possible to more correctly detect thewhole motion vector. Accordingly, the unintentional motion amount can bedetected with precision.

The above described objects and other objects, features, aspects andadvantages of the present invention will become more apparent from thefollowing detailed description of the present invention when taken inconjunction with the accompanying drawings.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a block diagram showing one embodiment according to thepresent invention;

FIG. 2 is a block diagram showing a motion detecting circuit of FIG. 1embodiment.

FIG. 3 is an illustrative view showing a principle of an electroniczooming and detection areas within an image field;

FIG. 4 is an illustrative view showing a principle of the electroniczooming and representative points and sampling points in the detectionareas;

FIG. 5 is an illustrative view showing a method for detecting an objecthaving repeated pattern by utilizing a pixel having a minimumcorrelative value and four pixels around the same;

FIG. 6 is an illustrative view showing a principle of an electronicpicture stabilization;

FIG. 7 is an illustrative view showing the detection areas within theimage field, to which a representative point matching method is applied;

FIG. 8 is a flowchart showing an operation of the embodiment;

FIG. 9 is a flowchart showing an operation succeeding to FIG. 8;

FIG. 10 is a flowchart showing a modified example of an operation fordetermining a whole motion vector; and

FIG. 11 is a graph showing a change of correlative values in respect tocoordinate positions for each image status in a case where therepresentative point matching method is utilized.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

A video camera 10 of a preferred embodiment shown in FIG. 1 includes asolid-state image sensing device 12 such as a CCD (Charge-CoupledDevice) which converts an optical signal being inputted from an object(not shown) through a lens 14 into an electrical signal. The electricalsignal from the solid-state image sensing device 12 is inputted to acamera circuit 16. As is well known, the camera circuit 16 includes asample-hold circuit by which the electrical signal from the solid-stateimage sensing device 12 is sampled and held. A level of the electricalsignal thus sampled and held is adjusted by an AGC (Automatic GainControl), and synchronization signals are added to the electrical signalby a synchronization signal adding circuit (not shown). Thus, the cameracircuit 16 converts the electrical signal from the solid-state imagesensing device 12 into an analog video signal. The analog video signalis further converted into a digital video signal by an A/D converter 18.The digital video signal is applied to a motion detecting circuit 20. Asthe motion detecting circuit 20, for example, an LSI "L7A0948"manufactured by Sanyo Electric Co., Ltd., an assignee of the presentinvention, may be utilized. Under control of a memory control circuit 22which is included in the same LSI constituting the motion detectingcircuit 20, the digital video signal is written into a field memory 24field-by-field.

The motion detecting circuit 20 evaluates, for each of four detectionareas A, B, C and D shown in FIG. 3, a position of one point having ahighest correlation degree (a minimum correlative value) and positionsof four points around the one point, and correlative values by utilizinga well-known representative point matching method. The position data andthe correlative value data are applied to a microcomputer 26.

More specifically, referring to FIG. 2, the motion detecting circuit 20shown in FIG. 1 includes an input end 28 which receives the digitalvideo signal from the A/D converter 18. The digital video signalinputted to the input end 28 is applied to the representative pointmemory 32 and a subtracting circuit 34, respectively, through a filter30. The filter 30 is a digital low-pass filter which is utilized forimprovement of an S/N ratio so as to secure a significant detectionaccuracy with a lesser number of representative points. Therepresentative point memory 32 stores position data and luminance dataof a plurality of representative points within each of the respectivedetection areas A-D shown in FIG. 3. In the embodiment shown, each ofthe detection areas is divided into thirty (30) regions, and therefore,thirty (30) representative points are determined, and accordingly, therepresentative point memory 32 stores the position data and theluminance data of the thirty (30) representative points. Each of thedivided regions 42 (FIG. 4) is constituted by 32 pixels in a horizontaldirection (X direction)×16 pixels in a vertical direction (Y direction).

The subtracting circuit 34 executes subtracting operations of theluminance data of the representative point of the last field read-outthe representative point memory 32 and luminance data of all the pixelsof the present field applied from the input end 28 via the filter 30,and obtains absolute values of subtraction results. That is, thesubtracting circuit 34 evaluates a luminance difference between theluminance data of the present field and the luminance data of the lastfield, and applies the luminance differences to an accumulating andadding circuit 36. The accumulating and adding circuit 36 executes anaccumulation and addition of the luminance differences of thirty (30) inthis embodiment obtained by the subtracting circuit 34 of the sameposition or pixel in the same region 42 so as to output correlativevalues data. The correlative values data is applied to an arithmeticoperation circuit 38 which evaluates a minimum correlative value andcalculates an average correlative value for each of the detection areasA-D, and evaluates position data of the pixel having the minimumcorrelative value. Data of the minimum correlative value, averagecorrelative value and positions thus obtained by the arithmeticoperation circuit 38 is applied to the above described microcomputer 26from an output end 40. In addition, such arithmetic operations for thecorrelative values can be performed by the above described LSI"L7A0948".

Then, in the microcomputer 26, a motion vector of a whole of a screen,i.e. the image field 44 (FIG. 3) (simply called as "whole motionvector") is calculated on the basis of the position data and thecorrelative value data.

First, a deviation of a pixel indicative of the minimum correlativevalue from the representative point is evaluated on the basis of theposition data of that pixel, and the deviation is made as a portionmotion vector. In addition, in order to provide accurate detection ofthe portion motion vector, an internal interpolation is performed byutilizing the correlative values of the four pixels around the pixelhaving the minimum correlative value so as to calculate the positiondata of the pixel having the minimum correlative value.

In addition, the microcomputer 26 further evaluates a propriety of theportion motion vector detected for each detection area, that is,determines whether each of the detection areas is a valid detection areaor an invalid detection area on the basis of the status of the image. Inthe embodiment shown, since the representative point matching method isutilized, the change of the correlative value with respect to thecoordinate positions becomes similar to FIG. 11 (described later).

Now, in order to determine whether or not the contrast of the screen islow; whether or not there is a moving object in the detection area; andwhether or not there is an object having a repeated pattern (stripes,for example) in the detection area, conditions (A)-(C) are defined asfollows.

(A) average correlative value>α;

(B) value obtained by dividing average correlative value by minimumcorrelative value>β;

(C) gradient>γ (when average correlative value≧ε); and

(D) gradient>δ (when average correlative value<ε);

In addition, α, β, γ, δ, and ε are constant threshold values, and γ≧δ.For example, the threshold values are set as α=36, β=7, γ=8, δ=4, andε=128.

The microcomputer 26 determines whether or not the contrast of thescreen is low in accordance with the above described condition (A). Themicrocomputer 26 further determines whether or not the moving objectexists within the detection area on the basis of the above describedcondition (B). Furthermore, the microcomputer 26 determines whether ornot the object having a repeated pattern exists in the detection area onthe basis of the above described condition (C) at a timing that theaverage correlative value is equal to or larger than the threshold valueε, or the above described condition (D) at a time that the averagecorrelative value is smaller than the threshold value ε. Theseprocessings are executed for each of the detection areas A-D so that itis determined whether or not the portion motion vector of each of thedetection areas A-D is erroneously detected due to the moving object orthe like other than unintentional motion and thus reliable, that is,whether or not each of the detection areas A-D is the valid detectionarea. If a detection area satisfies all the above described conditions(A)-(D), the detection area is determined to be the valid detectionarea, and if any one of the above described conditions is not satisfied,the detection area is determined to be an invalid detection area.

Determining, whether or not the detection area is the valid detectionarea is as follows:

At first, when the contrast of the screen is low, the luminancedifference is small, and therefore, the correlative value becomes small.When the entire screen is white, for example, the correlative value isvery small. In such a case, the reliability of the detection result issmall, and therefore, only when the condition (A) is satisfied, that is,the average correlative value>α, the detection area is determined as thevalid detection area. In addition, the threshold value α can bedetermined through field tests or examinations. Thus, on the basis ofthe average correlative value, it is determined whether or not thescreen is low contrast.

Furthermore, when the moving object exists in the detection area, thecorrelative value at a portion occupied by the moving object and thecorrelative value when no moving object exists are different from eachother. Various kinds of correlative values are obtained by the portionoccupied by the moving object, and the correlative value from thatportion becomes a large value generally (the correlation degree becomeslow). Therefore, when the moving object exists within the detectionarea, there is a possibility that the minimum correlative value becomeslarge, and the portion motion vector of that detection area may beerroneously detected. If the portion motion vector is erroneouslydetected, the whole motion vector is also erroneously detected; however,when the average correlative value is large, the portion motion vectoris reliable even if the minimum correlative value is large at someextent. On the other hand, when the average correlative value is small,the portion motion vector is reliable only when the minimum correlativevalue is smaller. Therefore, the detection area is determined as thevalid detection area when (the average correlative value/the minimumcorrelative value)>7, and if this condition is not satisfied, theportion motion vector of the detection area is not utilized so as toavoid the influence due to the above described erroneous detection.Thus, the microcomputer 26 determines the presence or absence of themoving object by evaluating the value of the average correlativevalue/the minimum correlative value.

Furthermore, in the microcomputer 26, one point having the minimumcorrelative value and the correlative values of four points around theone point are utilized so as to detect the object having repeatedpattern (stripes, for example).

More specifically, on the assumption that the minimum correlative valueis M, and the correlative values of the four points at left, right, up,and down are L, R, U, and D as shown in FIG. 5. The differences betweenthe respective correlative values, i.e. L-M, R-M, U-M and D-M arecalculated, and a minimum value of the differences is defined as thegradient. When the average correlative value is equal to or larger thanthe threshold value ε, the gradient is compared with the threshold valueγ that is determined through field tests. Then, if the gradient islarger than the threshold value γ, it is determined that the detectionarea is the valid detection area, and if the gradient is equal to orsmaller than the threshold value γ, the detection area is determined asthe invalid detection area. On the other hand, when the averagecorrelative value is smaller than the threshold value ε, the gradient iscompared with the threshold value δ that is determined through fieldtests. If the gradient is larger than the threshold value δ, thedetection area is determined as the valid detection area, and if thegradient is equal to or smaller than the threshold value δ, it isdetermined that the detection area is the invalid detection area.

Thus, in accordance with the conditions (A)-(D), it is determinedwhether or not each of the detection areas is the valid detection area.

Then, in accordance with the number of valid detection areas, the motionamount between the fields, i.e. the whole motion vector is determined.Therefore, the whole motion vector is representative of the amount ofmotion between the fields and a direction thereof.

The whole motion vector thus evaluated is applied to the memory controlcircuit 22. In the memory control circuit 22, a start address forreading-out the field memory 24 is determined on the basis of the wholemotion vector, and therefore, the digital video signal stores in thefield memory 24 becomes to be read-out at the start address. That is,the memory control circuit 22 moves an extracting area 46 (FIG. 6)formed by the digital video signal from the field memory 24 inaccordance with the whole motion vector calculated by the microcomputer26.

In addition, since the extracting area 46 cannot be moved by the digitalvideo signal read-out from the field memory 24 as it is, an electroniczooming circuit 48 (FIG. 1) is utilized.

With reference to FIG. 6, the electronic zooming circuit 48 (FIG. 1)defines the image extracting area 46 wherein an image is enlargedaccording to a zooming magnification with respect to the image field 44.A position of the image extracting area 46 can be freely moved within arange of the image field 44 by changing the start address forreading-out the digital video signal from the field memory 24. Then, inorder to obtain a video signal for a whole area of the image field 24 onthe basis of the digital video signal extracted from the imageextracting area 46, an image is enlarged by utilizing an internalinterpolation on the basis of the digital video signal read-out thefield memory 24.

Thus, by zooming-up an image of an arbitrary image extracting area 46within the image field 44 in an electronic manner by the electroniczooming circuit 48 (FIG. 1), a correctable range 50 that is equal to adifference between the image field 44 and the image extracting area 46can be formed.

If an unintentional motion occurs in the video camera 10 as shown inFIG. 7 according to a vibration of a hand of a person who operates thevideo camera 10, an image from the video camera 10 is blurred, resultingin a case where an object person exists in a left-lower portion withinthe image field 44 (shown at an upper portion in FIG. 7) or a case wherean object person exists at a right-upper portion within the image field(shown at a lower portion in FIG. 47. Therefore, by moving the imageextracting area 46 at every field according to the whole motion vectorthat is calculated by the microcomputer 26, as shown at a right portionin FIG. 7, the object person can be just positioned in the imageextracting area 46.

The digital video signal thus outputted from the electronic zoomingcircuit 48 is converted into an analog signal by a D/A converter 52 soas to be outputted from an output terminal 54.

In addition, the processing set forth in the following may be executedby the microcomputer 26.

At first, the microcomputer 26 evaluates a mean value of four portionmotion vectors (average vector) and a mean value of portion motionvectors of the valid detection areas, respectively.

Then, a dispersion is calculated by the microcomputer 26. A degree ofvariation of the portion motion vector of each of the detection areas isevaluated according to the dispersion. The dispersion is represented bythe following equations.

X direction dispersion=Σ (|X direction portion motion vectors ofdetection areas-X direction average vector|)/the number of the detectionareas; and

Y direction dispersion=Σ (|Y direction portion motion vectors ofdetection areas-Y direction average vector|)/the number of the detectionareas wherein, the X direction portion motion vector indicates an Xdirection component of the portion motion vector, and the Y directionportion motion vector indicates a Y direction component of the portionmotion vector, and the X direction average vector indicates an Xdirection component of the average vector, and the Y direction averagevector indicates a Y direction component of the average vector.

In accordance with the dispersion as described later, a dispersioncoefficient Hk is determined, and by multiplying the dispersioncoefficient Hk by the mean value of the portion motion vectors of thevalid detection areas, the whole motion vector is calculated. Inaddition, that the dispersion coefficient Hk is determined in the Xdirection and the Y direction, respectively, and the X directiondispersion coefficient is multiplied to the X direction component of themean value of the portion motion vectors of the valid detection areas,and the Y direction dispersion coefficient is multiplied to the Ydirection component of the mean value of the portion motion vectors.

(1) 0≦dispersion<8: Hk=1

(2) 8≦dispersion<16: Hk=0.75

(3) 16≦dispersion<24: Hk=0.5

(2) 24≦dispersion<32: Hk=0.25

In addition, in a case where the dispersion exceeds "32", a minimum oneof the four portion motion vectors is defined as the whole motionvector.

Thus, if there is a valid detection area, the whole motion vector iscalculated by multiplying the dispersion coefficient Hk to the meanvalue of the portion motion vectors of the valid detection areas;However, if there is no valid detection area, the whole motion vector isdefined as "the whole motion vector of the present field×a coefficientless than 1". In addition, as the coefficient less than 1, in theembodiment shown, "0.97" is utilized.

In addition, a reason why the dispersion coefficient is set stepwise inaccordance with the magnitude of the dispersion is as follows:

A fact that the dispersion is large indicates that the variation of theeach of respective portion motion vectors is large. This tendency occursin a case where a moving object exists at only a portion of the screen.On the other hand, a fact that the dispersion is small indicates thatthe variation of each of the respective portion motion vectors is small.This tendency occurs in a case where no moving object exists in thescreen, or a case where a moving object which moves all over the screenexists.

Therefore, if the whole motion vector is evaluated by utilizing theportion motion vectors of a case where the dispersion is large, thewhole motion vector having low reliability is obtained. On the otherhand, if the whole motion vector is evaluated by utilizing the portionmotion vectors of a case where the dispersion is small, the whole motionvector having high reliability is obtained. Accordingly, when thedispersion is large, the whole motion vector is made smaller bymultiplying a small dispersion coefficient Hk to the mean value of theportion motion vectors. Furthermore, when the dispersion is small, bymultiplying a large dispersion coefficient Hk to the mean value of theportion motion vectors, the whole motion vector is made larger. That is,the whole motion vector of the case where the dispersion is large is notused as possible, and in contrast, the whole motion vector of the casewhere the dispersion is small is used if possible, so that the wholemotion vector having high reliability can be obtained.

By utilizing the whole motion vector thus obtained, in accordance withthe above described manner, the unintentional motion is corrected, thatis, the electronic picture stabilization is performed.

With referring FIG. 8 to FIG. 10, a major operation in the microcomputer26 of the video camera 10 will be described. In addition, in a flowchartof FIG. 8, "n" ("1" to "4") is corresponding to the detection area A tothe detection area D.

In a step S1 shown in FIG. 8, "n" of the detection area is initially setas "1". If the average correlative value of the detection area n islarger than the threshold value α in a step S3, the process proceeds toa step S5 wherein it is determined whether or not the value obtained bydividing the average correlative value by the minimum correlative valueis larger than the threshold value β. If the value is larger than thethreshold value β in the step S5, in a step S7, it is determined whetheror not the average correlative value is equal to or larger than thethreshold value ε. If the average correlative value is equal to orlarger than the threshold value ε, in a step S9, it is determinedwhether or not the gradient of the detection area n is larger than thethreshold value γ. If "YES" is determined in the step S9, in a step S13,the detection area n is determined as a valid detection area. On theother hand, if the average correlative value is smaller than thethreshold value ε in the step S7, in a step S11, it is determinedwhether or not the gradient of the detection area n is larger than thethreshold value δ. If "YES" is determined in the step S11, in a stepS13, the detection area n is determined to be a valid detection area.

At a time that the average correlative value of the detection area n isequal to or smaller than the threshold value α in the step S3, a timethat the value obtained by dividing the average correlative value of thedetection area n by the minimum correlative value is equal to or smallerthan the threshold value β in the step S5, a time that the gradient ofthe detection area n is equal to or smaller than the threshold value γin the step S9, or a time that the gradient of the detection area n isequal to or smaller than the threshold value δ in the step S11, theprocess proceeds to a step S15 in which, the detection area n isdetermined as an invalid detection area.

After the step S13 or the step S15, in a step S17, "n" indicative of thedetection area is incremented by 1. Then, in a step S19, it isdetermined whether or not "n" is equal to or smaller than "4", and if"YES", the process returns to the step S3. That is, the above describedprocessings are repeated until "n" exceeds "4" in the step S19. If "n"exceeds "4" in the step S19, the process proceeds to a step S21 shown inFIG. 9.

In the step S21, the number of the valid detection areas is counted. Ina step S23, it is determined whether or not the number of the validdetection areas is "4", and if "NO", in a step S25, it is determinedwhether or not the number of the valid detection areas if "3". If "YES"is determined in the step S23 or the step S25, that is, if the number ofthe valid detection areas is more than "3", in a step S27, mean valuesare calculated of the X (horizontal, for example) direction portionmotion vectors and the Y (vertical, for example) direction portionmotion vectors are calculated for each of the valid detection areas.Then, in a step S29, by utilizing the mean values calculated in the stepS27, an absolute value of the X direction and an absolute value of the Ydirection are evaluated for each of the valid detection areas, and then,the X direction absolute value and the Y direction absolute value areadded to each other for each detection area to obtain the dispersion.The X direction absolute value is an absolute value of a differencebetween the X direction portion motion vectors of the respectivedetection areas and the mean value of the X direction portion motionvectors of the valid detection areas. Furthermore, the Y directionabsolute value is an absolute value of a difference between the Ydirection portion motion vectors of the respective detection areas andthe mean value of the Y direction portion motion vectors of the validdetection areas. Then, in a step S31, two dispersions having smallervalues are selected from the four dispersions calculated in the stepS29, and a mean value of the portion motion vectors of the detectionareas corresponding to the selected dispersions is calculated. Then, ina step S33, the mean value is used as the whole motion vector.

On the other hand, if "NO" is determined in the step S25, in a step S35,it is determined whether or not the number of the valid detection areasis "2". If the number of the valid detection areas is "2", in a stepS37, the portion motion vector of an arbitrary one of the invaliddetection areas is replaced with the whole motion vector evaluatedbefore one field, and then, the process proceeds to the step S27. Then,through the execution of the steps S27-S33, the whole motion vector isdetermined. In this embodiment shown, the portion motion vector of oneof the two invalid detection area is replaced by the whole motion vectorof the last field, but the portion motion vector of the remaininginvalid detection area is not replaced with the whole motion vector.Therefore, the whole motion vector is determined on the basis of theportion motion vectors of the two valid detection areas and the portionmotion vector of the invalid detection area being replaced with thewhole motion vector.

If it is determined that the number of the valid detection areas is not"2" in the step S35, in a step S39, it is determined whether or not thenumber of the valid detection areas is "1". If "YES" is determined inthe step S39, in a step S41, the portion motion vectors of arbitrary twoinvalid detection areas are replaced with the whole motion vectorevaluated one field before and the whole motion vector evaluated twofields before, respectively, and the process proceeds to the step S27.Then, through the execution of the steps S27-S33, the whole motionvector can be evaluated. In this embodiment shown, the portion motionvectors of the two invalid detection areas out of the three invaliddetection areas are replaced with the whole motion vector evaluated onefield before and the whole motion vector evaluated two field before,respectively, and for the remaining one invalid detection area, theportion motion vector is not replaced with the whole motion vector.Therefore, in this embodiment shown, the whole motion vector isdetermined on the basis of the portion motion vector of one validdetection area and the portion motion vectors of the two invaliddetection area being replaced with the whole motion vectors.

If the number of the valid detection areas is not "1", that is, it isdetermined that all the detection areas are invalid detection areas inthe step S39, in a step S43, by multiplying the coefficient "0.97" tothe whole motion vector evaluated one field before, the whole motionvector of the present field in the step S33 is obtained.

Thus, the unintentional motion is corrected by utilizing the wholemotion vector according to the number of the valid detection areas.However, a correction method itself is well known, and therefore, adetailed description thereof is omitted here.

In accordance with this embodiment, the determination whether or notthere is an object having repeated pattern within the detection areascan be made correctly, and the detection area is not determined as theinvalid detection area even if the contrast of the screen is slightlylowered. Therefore, the validation or the invalidation of the detectionareas can be correctly determined, and accordingly, the detectionaccuracy of the whole motion vector improves.

That is, according to the above described embodiment, the disadvantagesof the prior art set forth in the following can be eliminated.

In the prior art method wherein the unintentional motion component isdetected from the image information, the detection accuracy is largelyaffected by the status of the input image. Therefore, the propriety ofthe detected motion vector is evaluated by the microcomputer inaccordance with the status of the image.

FIG. 11 is a graph showing a change of a correlative value with respectto coordinate positions in a case where the representative matchingmethod is utilized. Normally, as shown by a solid line a in FIG. 11, thecorrelative value has a sharp minimum point indicative of the minimumcorrelative value, and the motion vector is determined on the basis ofthe position of the minimum correlative value. In contrast, if thecontrast of the screen is low, or if a moving object exists in thescreen, as shown by a one-dotted line b or a dotted line c in FIG. 11,the minimum correlative value cannot be accurately detected, andtherefore, the detection accuracy of the motion vector is lessened.Furthermore, in a case where there is an object having repeated pattern(striped image) in the screen, it becomes very difficult to detect themotion vector. In order to determine the above described image status,in the prior art method, the following conditions are set with respectto the correlative value:

(A) average correlative value>α;

(B) value obtained by dividing average correlative value by minimumcorrelative value>β; and

(C) gradient>γ

wherein, α, β and γ are constant threshold values, and for example, thesame are set as α=36, β=7 and γ=8.

In the prior art method, for each of the detection areas, it isdetermined whether or not the above described conditions (A)-(C) aresatisfied. Then, if any one of the conditions is satisfied, thedetection area is determined as an invalid detection area, and if allthe conditions are satisfied, the detection area is determined as avalid detection area. Then, a motion vector is determined by utilizingthe motion vectors obtained from the valid detection areas only. As amethod for determinating the motion vector, an averaging of the motionvectors of the valid detection areas is utilized, for example.

However, in the above described conditions of the prior art method, ifthe contrast of the screen is reduced, even when the condition (A) issatisfied, the condition (C) is not satisfied, and therefore, thethreshold value γ of the condition (C) is to made smaller. However, thethreshold value γ is made smaller irrespective of the contrast of thescreen, it may be erroneously determined whether or not there is anobject having a repeating pattern in the screen, and therefore, thedetection accuracy of the motion vector is dropped. In contrast, in theabove described embodiment, the gradient is compared with the largethreshold value or the small threshold value according to anothercondition, such a problem can be solved.

In addition, in the above described embodiment, the correlative value iscalculated through accumulation and addition of the luminancedifferences. However, the correlative value may be evaluated byutilizing a difference of the electric signals from the solid-stateimage sensing device 12 between the adjacent fields (frames) instead ofthe luminance differences.

Furthermore, in the step S31, only one dispersion may be selected, thatis, the arbitrary number of dispersions may be selected.

Furthermore, the coefficient utilized in the step S43 is not limited to"0.97", and therefore, an arbitrary coefficient having a value largerthan zero but smaller than 1 may be utilized.

Furthermore, instead of the steps S29 and S31 shown in FIG. 9, theprocessing shown in FIG. 10 may be executed. In this case, after theexecution of the step S27 shown in FIG. 9, the process proceeds to astep S45 shown in FIG. 10.

In the step S45, the dispersion is calculated. In calculating thedispersion, the X direction dispersion and the Y direction dispersionare independently calculated as indicated in the above describedequations. If 0≦dispersion<8 in a step S47, in a step S49, thedispersion coefficient Hk is set as "1" (Hk=1), and then, the processproceeds to a step S51. If "NO" is detected in the step S47, in a stepS53, it is determined whether or not 8≦dispersion<16. If 8≦dispersion<16in the step S53, in a step S55, the dispersion coefficient Hk is set as"0.75" (Hk=0.75), and then, the process proceeds to the step S51. If"NO" is detected in the step S53, and if 16≦dispersion<24 in a step S57,in a step S59, the dispersion coefficient Hk is set as "0.5" (Hk=0.5),and then, the process proceeds to the step S51. If "NO" is detected inthe step S57, and if 24≦dispersion<32 in a step S61, in a step S63, thedispersion coefficient Hk is set as "0.25" (Hk=0.25), and then, theprocess proceeds to the step S51.

In the step S51, "a mean value of the portion motion vectors of thevalid detection areas×Hk" is calculated. The process in the step S51 isperformed for the X direction portion motion vectors and the Y directionportion motion vectors, respectively, and in the step S33, the result ofthe step S51 for the X direction and the result of the step S45 for theY direction are synthesized with each other, whereby the whole motionvector is obtained.

On the other hand, "NO" is detected in the step S61, in a step S65, aportion motion vector having a minimum value is selected out of the fourportion motion vectors, and the same is utilized as the whole motionvector in the step S33 shown in FIG. 9.

In the video camera 10 which operates as shown in FIG. 10, even ifmoving objects exist in two of the four detection areas such as a lefthalf and a right half of the screen, that is, even if a moving objectexists at only a portion of the screen, the influence due to the movingobjects becomes small, and therefore, it is possible to determine thewhole motion vector with precision.

In addition, irrespective of the valid detection area or the invaliddetection area, the whole motion vector which is obtained by multiplyingthe dispersion coefficient Hk by the average value of the portion motionvectors of the detection areas. This becomes effective for a case whereno determination of either the valid detection area of the invaliddetection area is performed.

Furthermore, the dispersion may be compared with arbitrary values andthe values of the dispersion coefficients are not limited to the abovedescribed ones.

Furthermore, the four detection areas are defined within the image fieldin the above described embodiments. However, the number of the detectionareas is arbitrary, and the greater the number of the detection areas,the better the detection accuracy.

Furthermore, in the above described embodiments, the luminancedifference is evaluated between the adjacent fields; however, the samemay be evaluated between the adjacent frames. Furthermore, thereplacement of the whole motion vector and may be performed not a uniton a frame basis rather than on a field basis.

Although the present invention has been described and illustrated indetail, it is clearly understood that the same is by way of illustrationand example only and is not to be taken by way of limitation, the spiritand scope of the present invention being limited only by the terms ofthe appended claims.

What is claimed is:
 1. A video camera, comprising:portion motion vector detecting means for detecting a portion motion vector of each of a plurality of detection areas formed in an image field; valid detection area determining means for determining whether or not each of the detection areas is a valid detection area through determination whether or not each of the portion motion vectors is effective for detecting an unintentional motion of the camera; and whole motion vector detecting means for detecting a whole motion vector in accordance with methods different from each other in accordance with the number of the valid detection area determined by said valid detection area determining means, wherein said whole motion vector detecting means comprises:first dispersion calculating means for obtaining a first dispersion on the basis of the respective portion motion vectors; selection means for selecting the arbitrary number of the first dispersions having smaller values; and first detecting means for detecting the whole motion vector on the basis of the portion motion vectors of the detection areas corresponding to the selected first dispersions.
 2. A video camera according to claim 1, wherein said first dispersion calculating means comprises:first calculating means for evaluating a first absolute value representative of a difference between the portion motion vectors in a horizontal direction of each of the detection areas and a mean value of the portion motion vectors in the horizontal direction of the valid detection areas; second calculating means for evaluating a second absolute value representative of a difference between the portion motion vectors in a vertical direction of each of the detection areas and a mean value of the portion motion vectors in the vertical direction of the valid detection areas; and adding means for obtaining said first dispersion by adding the first absolute value and the second absolute value to each other for each of the detection areas.
 3. A video camera according to claim 1 or 2, wherein said first detecting means detects the whole motion vector by averaging the portion motion vectors.
 4. A video camera according to any one of claims 1, 2 or 3, wherein the number of the detection areas is four, and said selection means selects two first dispersions.
 5. A video camera according to claim 1, wherein said whole motion vector detecting means comprises:first replacement means for replacing the portion motion vector of the invalid detection area with the whole motion vector before the present field or frame when the number of valid detection areas is less than a predetermined threshold value; and second detection means for detecting the whole motion vector on the basis of the portion motion vector of the invalid detection area being replaced with the whole motion vector and the portion motion vectors of the valid detection areas.
 6. A video camera according to claim 5, wherein said first replacement means replaces the portion motion vector of the invalid detection area with the whole motion vector evaluated one field or one frame before.
 7. A video camera according to claim 1, wherein said whole motion vector detecting means comprises:second replacement means for replacing the portion motion vectors of the invalid detection areas with a plurality of whole motion vectors which are different from each other and evaluated before the present field or frame when the number of valid detection areas is less than a predetermined threshold value; and third detection means for detecting the whole motion vector on the basis of the portion motion vectors of the invalid detection areas being replaced with the whole motion vectors and the portion motion vector of the valid detection area.
 8. A video camera according to claim 7, wherein said second replacement means replaces the portion motion vectors of the invalid detection areas with the whole motion vectors evaluated one field or frame before and two fields or frames before.
 9. A video camera according to claim 1, wherein said whole motion vector detection means includes fourth detection means for detecting the whole motion vector of the present field or frame by multiplying a predetermined coefficient to the whole motion vector evaluated one field or frame before when the number of valid detection areas is zero.
 10. A video camera according to claim 9, wherein said coefficient is a numeral value smaller than
 1. 11. A video camera according to claim 10, wherein said coefficient is 0.97.
 12. A video camera according to claim 1, wherein said valid detection area determining means comprises:first means for evaluating correlative values of pixels through comparison of an image signal of the last field or frame and an image signal of the present field or frame; means for evaluating an average correlative value that is a mean value of the correlative values; means for evaluating a minimum correlative value that is a minimum value of the correlative values; means for evaluating a value obtained by dividing the average correlative value by the minimum correlative value; means for evaluating a gradient associated with the minimum correlative value; and means for determining whether or not the portion motion vector is correctly detected in accordance with the following conditions:(A) average correlative value>first threshold value; (B) value obtained by dividing average correlative value by minimum correlative value>second threshold value; (C) gradient>third threshold value (when average correlative value≧fifth threshold value; and (D) gradient>fourth threshold value (when average correlative value<fifth threshold value), said third threshold value being equal to or larger than the fourth threshold value.
 13. A video camera according to claim 1, wherein said whole motion vector detecting means comprises:second dispersion calculating means for obtaining a second dispersion on the basis of the portion motion vectors; comparing means for comparing the second dispersion with a predetermined threshold value; and fifth detection means for detecting the whole motion vector by methods different from each other in accordance with a comparison result by the comparing means.
 14. A video camera according to claim 13, wherein said second dispersion calculating means comprises:averaging means for evaluating an average vector that is a mean value of the portion motion vectors; and calculating means for evaluating the second dispersion by adding absolute values of differences between the portion motion vectors and the average vector to each other, and by dividing an addition result by the number of absolute values.
 15. A video camera according to claim 14, wherein said portion motion vector detecting means includes means for detecting X direction portion motion vectors and Y direction portion vectors, wherein said averaging means includes means for evaluating an X direction average vector and a Y direction average vector, respectively, and wherein said calculating means evaluates the second dispersion of X direction component of the portion motion vector and the second dispersion of Y direction component of the portion motion vector.
 16. A video camera according to claim 13, wherein said comparison means compares a plurality of threshold values with the second dispersion.
 17. A video camera according to claim 13, wherein said fifth detecting means comprises:coefficient setting means for setting a dispersion coefficient in accordance with a comparison result by the comparing means; and means for obtaining the whole motion vector by multiplying the dispersion coefficient to a mean value of the portion motion vectors of the valid detection areas.
 18. A video camera according to claim 13, wherein said fifth detecting means includes means for regarding a minimum portion motion vector as the whole motion vector when the second dispersion is larger than a predetermined value.
 19. The video camera of claim 1, wherein said portion motion vector detecting means includes:first means for evaluating correlative values of pixels through comparison of an image signal of the last field or frame and an image signal of the present field or frame, means for evaluating an average correlative value that is a mean value of the correlative values, means for evaluating a minimum correlative value that is a minimum value of the correlative values, means for evaluating a value obtained by dividing the average correlative value by the minimum correlative value, means for evaluating a gradient associated with the minimum correlative value, and means for determining whether or not the portion motion vector is correctly detected in accordance with the following conditions:(A) average correlative value>first threshold value; (B) value obtained by dividing average correlative value by minimum correlative value>second threshold value; (C) gradient>third threshold value (when average correlative value≧fifth threshold value; and (D) gradient>fourth threshold value (when average correlative value<fifth threshold value), said third threshold value being equal to or larger than the fourth threshold value. 